Design : ibex

Platform: sky130hd

A sample dataset with multiple runs varying the design utilization and the layer_adjust parameters for the routing layers. All design metrics from the runs are collected for analysis.

Metrics data is represented as json files in the METRICS2.1 format. Each experiment in the run is a separate json file. All of the files are read into a DataFrame 'json_df'.

• Rows of the DataFrame represent an experiment.
• Columns of the DataFrame represent the metrics.

Number of runs in the dataset: 6699
Number of metrics in each data: 87

• The columns in the DataFrame, which are the metrics colleced for each run.

Index(['run.flow__generate__date', 'run.flow__metrics__version',
       'run.flow__openroad__version', 'run.flow__openroad__commit',
       'run.flow__scripts__commit', 'run.flow__uuid', 'run.flow__design',
       'run.flow__platform', 'run.flow__platform__commit', 'run.flow__variant',
       'synth.design__instance__stdcell__count',
       'synth.design__instance__stdcell__area', 'synth.runtime__total',
       'synth.cpu__total', 'synth.mem__peak', 'constraints.clocks__count',
       'constraints.clocks__details', 'floorplan.timing__setup__tns',
       'floorplan.timing__setup__ws',
       'floorplan.design__instance__stdcell__area',
       'floorplan.design__instance__design__util',
       'floorplan.design__io__count',
       'floorplan.design__instance__macros__count', 'floorplan.runtime__total',
       'floorplan.cpu__total', 'floorplan.mem__peak',
       'globalplace.route__wirelength__estimated',
       'globalplace.timing__setup__tns', 'globalplace.timing__setup__ws',
       'globalplace.runtime__total', 'globalplace.cpu__total',
       'globalplace.mem__peak', 'placeopt.timing__setup__tns',
       'placeopt.timing__setup__ws', 'placeopt.design__instance__design__area',
       'placeopt.design__instance__design__util',
       'placeopt.design__instance__stdcell__count', 'placeopt.runtime__total',
       'placeopt.cpu__total', 'placeopt.mem__peak',
       'detailedplace.timing__setup__tns', 'detailedplace.timing__setup__ws',
       'detailedplace.design__instance__displacement__total',
       'detailedplace.design__instance__displacement__mean',
       'detailedplace.desgin__instance__displacement__max',
       'detailedplace.route__wirelength__estimated',
       'detailedplace.runtime__total', 'detailedplace.cpu__total',
       'detailedplace.mem__peak', 'cts.clock__latency__min',
       'cts.clock__latency__max', 'cts.clock__skew__worst',
       'cts.timing__setup__tns__prerepair', 'cts.timing__setup__ws__prerepair',
       'cts.timing__setup__tns', 'cts.timing__setup__ws',
       'cts.design__instance__hold_buffer__count',
       'globalroute.clock__latency__min', 'globalroute.clock__latency__max',
       'globalroute.clock__skew__worst', 'globalroute.timing__setup__tns',
       'globalroute.timing__setup__ws', 'globalroute.timing__clock__slack',
       'globalroute.timing__clock__period', 'globalroute.runtime__total',
       'globalroute.cpu__total', 'globalroute.mem__peak',
       'detailedroute.route__wirelength', 'detailedroute.route__via__count',
       'detailedroute.route__drc_errors__count',
       'detailedroute.runtime__total', 'detailedroute.cpu__total',
       'detailedroute.mem__peak', 'finish.clock__latency__min',
       'finish.clock__latency__max', 'finish.clock__skew__worst',
       'finish.timing__setup__tns', 'finish.timing__setup__ws',
       'finish.power__internal__total', 'finish.power__switch__total',
       'finish.power__leakage__total', 'finish.power__total',
       'finish.design__instance__area', 'finish.design__instance__utilization',
       'finish.runtime__total', 'finish.cpu__total', 'finish.mem__peak'],
      dtype='object')

Data Engineering

• Extract only the relevant columns -- for this exercise, get the design name, flow variant and route metrics.
• Parse Variant and extract the relevant features -- Utilization, layer adjust for each routing layer.
• Rename column names.
• Create new columns (features) based on other column values -- for e.g. Success/Fail for the router.

• Print the head of the head of the data frame. As we can see, all of the parameter values used in the experiment is in the "Variant" string. We will have to parse the string and create individual columns for the features we are interested in.

	Design	Variant	GR WNS	Wire Length	Vias	DRC Errors	Route Runtime
0	ibex	CORE_UTIL_22-AR_1-DIEtoCORE_10-GP_PAD_2-LAYER_ADJUST_0.5-M1_0.2-M2_0.1-M3_0.125-M4_0.2-M5_0.5-GR_OVERFLOW_0	-2.14	910717	140916	0	52:22.64
0	ibex	CORE_UTIL_30-AR_1-DIEtoCORE_10-GP_PAD_2-LAYER_ADJUST_0.5-M1_0.15-M2_0.2-M3_0.2-M4_0.175-M5_0.5-GR_OVERFLOW_0	-1.82	815136	142411	0	1:17:35
0	ibex	CORE_UTIL_34-AR_1-DIEtoCORE_10-GP_PAD_2-LAYER_ADJUST_0.5-M1_0.2-M2_0.2-M3_0.175-M4_0.175-M5_0.5-GR_OVERFLOW_0	-1.63	783250	141540	0	2:53:06
0	ibex	CORE_UTIL_24-AR_1-DIEtoCORE_10-GP_PAD_2-LAYER_ADJUST_0.5-M1_0.175-M2_0.1-M3_0.175-M4_0.125-M5_0.5-GR_OVERFLOW_0	-2.27	870864	137421	0	1:05:53
0	ibex	CORE_UTIL_40-AR_1-DIEtoCORE_10-GP_PAD_2-LAYER_ADJUST_0.5-M1_0.2-M2_0.125-M3_0.15-M4_0.15-M5_0.5-GR_OVERFLOW_0	N/A	ERR	ERR	ERR	ERR

• Extract individual features from the "Variant" string.
• Print the head of the dataframe after extracting the features and performing further data engineering. This data frame is now ready to be used. Notice that we have also created additional columns for capturing whether Global Route and Detailed Route completed. We have also caluclated a 'Weighted Adjust' column that is a simple mean of the 'layer adjust' for each of the routing layers.

	Design	Variant	GR WNS	Wire Length	Vias	DRC Errors	Route Runtime	GR Success	Success	Core Util	M1 Adjust	M2 Adjust	M3 Adjust	M4 Adjust	M5 Adjust	Weighted Adjust
0	ibex	CORE_UTIL_22-AR_1-DIEtoCORE_10-GP_PAD_2-LAYER_ADJUST_0.5-M1_0.2-M2_0.1-M3_0.125-M4_0.2-M5_0.5-GR_OVERFLOW_0	-2.14	910717	140916	0	52:22.64	True	True	22.0	0.200	0.100	0.125	0.200	0.5	0.225
0	ibex	CORE_UTIL_30-AR_1-DIEtoCORE_10-GP_PAD_2-LAYER_ADJUST_0.5-M1_0.15-M2_0.2-M3_0.2-M4_0.175-M5_0.5-GR_OVERFLOW_0	-1.82	815136	142411	0	1:17:35	True	True	30.0	0.150	0.200	0.200	0.175	0.5	0.245
0	ibex	CORE_UTIL_34-AR_1-DIEtoCORE_10-GP_PAD_2-LAYER_ADJUST_0.5-M1_0.2-M2_0.2-M3_0.175-M4_0.175-M5_0.5-GR_OVERFLOW_0	-1.63	783250	141540	0	2:53:06	True	True	34.0	0.200	0.200	0.175	0.175	0.5	0.250
0	ibex	CORE_UTIL_24-AR_1-DIEtoCORE_10-GP_PAD_2-LAYER_ADJUST_0.5-M1_0.175-M2_0.1-M3_0.175-M4_0.125-M5_0.5-GR_OVERFLOW_0	-2.27	870864	137421	0	1:05:53	True	True	24.0	0.175	0.100	0.175	0.125	0.5	0.215
0	ibex	CORE_UTIL_40-AR_1-DIEtoCORE_10-GP_PAD_2-LAYER_ADJUST_0.5-M1_0.2-M2_0.125-M3_0.15-M4_0.15-M5_0.5-GR_OVERFLOW_0	N/A	ERR	ERR	ERR	ERR	False	False	40.0	0.200	0.125	0.150	0.150	0.5	0.225

• Print the statistics for each of the entries in the dataframe.

	count	mean	std	min	25%	50%	75%	max
GR WNS	6076.0	-1.996942	0.392577	-2.74	-2.270	-2.13	-1.630	-1.42
Wire Length	6076.0	855751.097762	62264.280045	782595.00	797771.000	846315.00	911142.000	966329.00
Vias	6076.0	143527.524687	6869.350589	137185.00	140294.000	141366.00	142559.500	164724.00
DRC Errors	6076.0	1.335912	5.073183	0.00	0.000	0.00	0.000	34.00
Core Util	6076.0	28.984529	5.739511	20.00	24.000	28.00	34.000	38.00
M1 Adjust	6076.0	0.149922	0.035571	0.10	0.125	0.15	0.175	0.20
M2 Adjust	6076.0	0.149979	0.035603	0.10	0.125	0.15	0.175	0.20
M3 Adjust	6076.0	0.149971	0.035569	0.10	0.125	0.15	0.175	0.20
M4 Adjust	6076.0	0.149979	0.035595	0.10	0.125	0.15	0.175	0.20
M5 Adjust	6076.0	0.500000	0.000000	0.50	0.500	0.50	0.500	0.50
Weighted Adjust	6076.0	0.219970	0.014222	0.18	0.210	0.22	0.230	0.26
Runtime	6076.0	9299.832620	10843.228411	2347.00	3910.000	5144.00	9016.250	68530.00

• We can observe a huge variation in runtime for the different runs and also a substantial variation in the routed wirelength. We would like to observe the relation between the various parameters on both runtime and routed wirelength.

Print some of base metrics from the data set.

• Number of success/failures.
• Min, Max of Wirelength, number of vias, drc errors.

Number of successful runs: 6076
Number of failed runs: 623

Min Wire Length: 782595.0,  Max Wire Length: 966329.0

Min Vias: 137185,  Max Vias: 164724

Min DRC Errors: 0,  Max DRC Errors: 34

• Plot the number of successful runs and number of failures with respect to core utlization. We can see that the trends are what we would expect. As utilization increases, the number of successful runs decreases and the number of failure runs increases.

• We can observe that at utilization values of above 38, there is a drastic increase in the number of failures. We also observe that for the range of layer_adjust settings that we are using, it does not have a direct impact on the number of successful and doomed runs. Choosing higher layer adjust values will show a more direct correlation to success/doomed runs.
  

• Generate a scatter plot of Wirelength Vs Core Utilization. We can see the trend where the wirelength decreases with increased utilization. However, at a certain utilization value the wirelength starts to increase due to the router trying more detours to resolve DRC errors.

• Let us now examine how the route runtime varies with utilization.

• We can see that the runtime is fairly flat for smaller utilizations. As the utilization value goes above 35 the runime starts to degrade and rise exponentially.

• Let's now examine the number of DRC errors.

• Once again, we see an expected trend -- the number of DRC errors is 0 for lower utilizations and starts to increase with utilizations above 35.

Logistic Regression

• Now let use build a logistic regression model to predict successful or dommed runs for this design based on input parameters of utilization and layer adjust values.

LogisticRegression()

• Print the confusion matrix for the trained model

array([[ 135,    3],
       [   0, 1202]])

Model Accuracy: 0.9978
Model Precision: 0.9975
Recall Score: 1.0000

• The above model is very accurate. Now check to see the accuracy of the Weighted Adjust parameter as the sole predictor

[[ 135    3]
 [   0 1202]]
Model Accuracy: 0.9978
Model Precesion: 0.9975
Recall Score: 1.0000

• For this dataset, the weighted layer_adjust seem to have similar accuracy as the individual layer adjusts, as within the range of the layer adjusts, the core utilization has more direct impact on success or failure.

• Now for runs that are successful, let us predict the wirelength based on parameters using a simple linear regression.

Predicted Wire Length for input parameters: [34, 0.1, 0.1, 0.1, 0.1, 0.5] is 837732.01
Actual Wire Length: 785088.0

[NbConvertApp] Converting notebook METRICS2.1-fine-grain.ipynb to html
[NbConvertApp] Writing 577812 bytes to METRICS2.1-fine-grain.html